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(54) System and Method for content-based multinnedia retrieval 



(57) The present Invention relates to a multinnedia 
retrieval system and a method thereof which is capable 
of performing a multimedia data retrieval anrtong differ- 
ent systems using color histograms constructed with dif- 
ferent color spaces and color quantization methods, in 
particular to a content-based multimedia retrieval sys- 
tem and a method thereof which is capable of retrieving 
multimedia data among different systems, by extracting 
or converting a color histogram of a query image or a 
color histogram of an image to be retrieved into a color 
histogram of the same color space and color quantiza- 
tion method each other, and using the extracted or con- 
verted color histogram. 
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Description 

BACKGROUND OF THE INVENTriON 

5 1 . Field of the Invention 

[0001] The present invention relates to a system for retrieving multimedia data and a method thereof, In particular 
to a content-based multimedia retrieval system and a method thereof which is capable of retrieving muitimedia data 
among different systems using color histograms constructed with different color spaces and color quantization methods. 

10 

2. Description of the Prior Art 

[0002] In retrieval of content-based multimedia data in the conventional technology, color distribution of an image is 
displayed as a color histogram, the color histogram is defined in accordance with a color space and a color quantization 
IS method. 

[0003] In addition, in retrieval of content-based multimedia data in the conventional technology, the color histogram 
is mainly used. 

[0004] In other words, it is possible to perfomi an image or a video retrieval based on colors among systems using 
color histograms constructed with same color spaces and quantization methods on the internet circumstances. On the 
20 contrary, it is impossible to perform the image or video retrieval among systems using color histograms constructed 
with different color spaces and different quantization methods each other 

[0005] Herein, the color histogram displays the color distribution of the image, is easy to calculate similarity for com- 
paring the color histograms, has a character unconcerned in position of a color pixel (Sometimes, a character concerned 
in position of a color pixel is needed in an image retrieval. For that, other feature display is used.), and has a better 
25 performance than the other color display method (texture histogram) for content-based image retrieval. 

[0006] The content-based multimedia retrieval system using the color histogram will now be described with reference 
to acconDpanying FIG.1. 

[0007] FIG.1 is a block diagram illustrating the construction of the content-based multimedia retrieval system using 
color histograms in accordance with the conventional technology. 

30 [0008] As depicted in FIG.1 , the content-based multimedia retrieval system using the color histogram comprises a 
color quantizer 100 for being Inputted query image inputted by a user and extracting a color histogram as feature 
information of the query image, an image database 103 for storing a plurality of retrieval object image data 103-1 - 
1 03-N, a feature database 1 02 for storing the color histograms 1 02-1 - 1 02-N as the feature inf omnation of the plurality 
of retrieval object images, and a retrieval unit 101 for calculating similarity between the color histogram of the query 

35 image and the color histograms of the plurality of retrieval object Images and outputting an image in accordance with 
the similarity as a retrieval result. 

[0009] The operation of the content-based multimedia retrieval system using the color histogram in accordance with 
the conventional technology will now be described. 

[0010] First, the color quantizer 1 00 extracts the color histograms 102-1 - 1 02-N in order to find the feature infor- 
^ mation of the image data 103-1 - 1 03-N about the plurality of retrieval object images. In other words, the color quantizer 
1 00 extracts one color histogram 1 02-1 per each image 1 03-1 . 

[0011] The extracted color histograms 102-1 - 1 02-N are connected to the pertinent images 103-1 - 103-N.andare 
stored on the feature database 102. In other words, the extracted color histograms 102-1 - 1 02-N stored on the feature 
database 1 02 display the color distribution of the each image. 
45 [0012] In addition, when the query image is inputted by the user, the color quantizer 1 00 extracts the color histogram 
about the query image. 

[0013] In other words, the color quantizer 1 00 quantizes the designated color space into a plurality of subordinate 
spaces in accordance with the designated color space and quantization method. 

[0014] Herein, the each quantized subordinate space is matched to one bin, color values of all pixels in a certain 
50 image construct a color histogram in proportion to a frequency of occun-ence con-esponding to the each subordinate 
space. 

[0015] The retrieval unit 101 calculates the similarity of the color histograms 102-1 - 102-N displaying the color 
histogram of the extracted query image and the color distribution of the retrieval object images 1 03 - 1 03-N stored in 
the feature database 102, and outputs the retrieval result in accordance with the similarity. 
55 [0016] For example, the retrieval unit 101 calculates the similarity of the histograms 102-1 - 102-N extracted in 
advance for the image retrieval based on the color and stored in the feature database 1 02 and the histogram of the 
query image by using a known nnethod, namely, a histogram matching method such as a histogram intersection, and 
outputs the each image con-esponding to the color histogram as a retrieval result on the basis of the magnitude of the 
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calculated simllanty value. 

[0017] Herein, when the color histogram of the query image is extracted in advance and is stored in the feature 
database 1 02, the retrieval unit 1 01 calculates the simllarrty between the color histogram of the query image extracted 
in advance and the color histograms of the retrieval object images, and outputs an image in accordance with the 
5 similarity as a retrieval result (There is no need to extract newly the color histogram of the query image by the color 
quantizer). 

[0018] On the contrary, when the query image is not stored on the feature database 102 after being extracted in 
advance, the color quantizer 1 00 extracts newly the color histogram of the query image, and provides it to the retrieval 
unit 101. 

10 [0019] However, in the conventional content-based multimedia retrieval system using the color histogram, it is pos- 
sible to compare the color histogranns among different systems only when the same color space and same color quan- 
tization are adapted to the color quantizers. 

[0020] For example, when the color histograms constructed with same color space and same quantization method 
are used between same two systems on the intemet circunnstances, it is possible to perform the multimedia data 
'5 retrieval based on the colors. 

[0021] On the contrary, between two different systems, when color histograms constructed with color spaces and 
quantization method are not same. It is impossible to periomn the multimedia data retrieval based on the colors. It will 
now be described with reference to accompanying FIG.2. 

[0022] FIG.2 illustrates multimedia retrieval relations between different systems in accordance with the conventional 
20 technology. 

[0023] As depicted In FIG.2, because the meaning of each bin of the color histogram is changed by different color 
space and different color quantization, comparing a color histogram used in a local system A 200 with a color histogram 
used in a local system B is impossible. 

[0024] For example, when color histograms 201 -1 * 201-N of images are stored in the feature database 201 of the 
25 local system A 200, a color histogram having 1 25 (5x5x5) bin number is constructed by perfonnning 5 levels of unrfomi 
quantization about each RGB element by using a RGB (Red Green Blue) color space 202. 

[0025] When color histogranns 20&-1 ~ 205-N of images are stored on the feature database 205, the local system B 
204 constructs and stores a color histogram having 343 (7X7X7) bin number by performing 7 levels of unifonn quan- 
tization about each HSV (Hue Saturation Value) elenncnt by using a HSV (Hue Saturation Value) color space 206. 
30 [0026] Accordingly, it is impossible to compare the color histogram of the local system A with the color system of the 
local system B. 

[0027] As described above, in the conventional technology, it is impossible to perform the multimedia data retrieval 
between the systems using the color histogranns constructed with the different color spaces and color quantization 
methods each other. 

35 [0028] In addition, in the conventional technology, it is Impossible to retrieve a retrieval object Image about a query 
image of itself in a database of the other party between the systenns using the color histograms constructed with the 
different color spaces and color quantization methods. 

[0029] In addition, in the conventional technology, it is impossible to connect the databases which store various, 
different, wide range of innages to a retrieval unit in a network circumstances such as the intemet in order to retrieve 
40 images. 

SUMI^ARY OF THE INVENTION 

[0030] Accordingly, it would be desirable to provide a content-based multimedia retrieval system and a method thereof 
45 which is capable of perfonning a muftimedia data retrieval anrrong different systenns using color histograms constructed 
with different color spaces and different color quantization methods. 

[0031] It would also be desirable to provide the content-based multimedia retrieval system and the method thereof 
which is capable of retrieving an image to be retrieved corresponding to a query image of itself in a database of the 
other party, when retrieving multimedia data in systenns using color histograms constructed with different color spaces 
50 and color quantization methods. 

[0032] it would also be desirable to provide the content-based multimedia retrieval system and the method thereof 
which is capable of retrieving images by connecting databases storing various, different, wide range of images to a 
retrieval unit on a network circumstances such as the intemet. 

[0033] It would also be desirable to provide the content-based multimedia retrieval system and the method thereof 
55 whfch is capable of increasing multimedia data retrieval speed by extracting a color histogram of a query image in 
advance and storing the extracted color histogram of the query image. 

[0034] The invention provides a content-based muftimedia retrieval system for retrieving content-based multimedia 
data comprising a first color quantization means for extracting a color histogram of query multimedia data; a second 
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color quantization means for extracting color histograms of multimedia data to be retrieved ; and a histogram conversion 
means for converting the color histogram of the extracted query multimedia data and the color histogram of the multi- 
media data to be retrieved so as to be same each other. 

[0035] The invention also provides a content-based multimedia retrieval method for retrieving multimedia data by 
5 comparing query multimedia data with multimedia data to be retrieved comprising inputting query multimedia data; 
converting the color histogram of the inputted query multimedia data and the color histogram of the multimedia data 
to be retrieved so as to be same each other; and calculating a similarity between the query multimedia data and 
multimedia data to be retrieved on the basis of the converted color histogram and outputting a retrieval result in ac- 
cordance with the calculated similarity. 
10 [0036] The invention also provides a content-based multimedia retrieval method for retrieving multimedia data by 
comparing query multimedia data with retrieval object multimedia data comprising extracting a color histogram of the 
query multimedia data; extracting a color histogram of the multimedia data to be retrieved; comparing the extracted 
color space and color quantization method of the query image with the color space and color quantization method of 
the multimedia data to be retrieved ; and converting the color histogram of the extracted query multimedia data or the 
IS multimedia data to be retrieved into the same color histogram so as to have same color space and color quantization 
method each otiier when the color spaces and color quantization methods of the extracted multimedia data and mul- 
timedia data to be retrieved are different each other, and perf onning a retrieval in accordance with the similarity between 
the query multimedia data and multimedia data to be retrieved. 

[0037] The invention also provides, in a content-based multimedia retrieval system and a method thereof in accord- 
20 ance with the present invention, a multimedia data generation method comprising generating color space description 
information describing what color space the color histogram of the multimedia data is constructed; and generating 
quantization description information describing what color quantization method the color histogram of the multimedia 
data is constructed. 

[0038] The invention also provides a histogram conversion method of multimedia data in a content-based multimedia 
25 retrieval on the basis of the color histogram as a feature element for content-based multimedia retrieval comprising 
projecting each subordinate color space of the color space of the quantized query multimedia data on the color space 
of the multimedia data to be retrieved; and distributing a bin value to each quantum of the color space of the multimedia 
data to be retrieved, which is overiapped with the projected color spaces. 

[0039] The invention also provides a content-based multimedia retrieval system comprising a first color quantization 
30 means for extracting color histogram of the multimedia data to be retrieved; a description means for descn'bing color 
space and color quantization method of the extracted color histogram; and a second color quantization means for 
extracting color histogram of a query multimedia data with a method same as the described color space and color 
quantization method in order to perform the multimedia data retrieval. 

[0040] The invention also provides, in the retrieval method for retrieving the multimedia data to be retrieved with an 
35 input of the query multimedia data, a content-based multimedia retrieval method comprising judging whether the color 
histogram of the query multimedia data corresponding to the color space and quantization method of the multimedia 
data to be retrieved is stored before; and calculating a similarity between the color histogram of the stored query 
multimedia data and the color histogram of the multimedia data to be retrieved and perfomning a multimedia retrieval 
in accordance with the calculated similarity. 

40 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0041] FIG.1 illustrates a construction of a content-based multimedia retrieval system using a color histogram in 
accordance with the conventional technology. 
45 [0042] FIG.2 illustrates multinr^edia retrieval relations between different systems in accordance with the conventional 
technology. 

[0043] FIG.3 illustrates a construction of a first embodiment of a content-based multimedia retrieval system in ac- 
cordance with the present invention. 

[0044] FIG.4 illustrates multimedia retrieval relations between different systems in accordance with the present tn- 
50 vention. 

[0045] FIG.5 is a flow chart illustrating a first embodiment of a content-based multimedia retrieval method In accord- 
ance with the present invention. 

[0046] FIG.6 illustrates a color histogram conversion method in accordance with the first embodiment of the present 
invention. 

55 [0047] FIG.7 is a flow chart illustrating a first enU>odiment of an operation process of a color histogram converter in 
accordance with the present invention. 

[0048] FIG.8 illustrates a color histogram conversion method using a sampling method in accordance with the present 
invention. 
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[0049] FIG.9 is a flow chart illustrating a second embodiment of an operation process of the color histogram converter 
in accordance with the present invention. 

[0050] FiG.1 0 illustrates a second embodiment of the content-based multimedia retrieval system in accordance with 
the present invention. 

5 [0051] FIG:11 is a flow chart illustrating a second embodiment of the content-based muKmedia retrieval method in 
accordance with the present invention. 

[0052] FIG.12 is a flow chart illustrating a third embodiment of the content-based multimedia retrieval method in 
accordance with the present invention. 

10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0053] The present invention is to provide a content-based multimedia retrieval system and a method thereof which 
is capable of perfomning a multimedia data retrieval among different systems having multimedia data constructed with 
different color spaces and different color quantization methods (systems using color histograms constructed with dif- 
is ferent color spaces and color quantization methods). 

[0054] First, a first embodiment of the content-based multimedia retrieval system in accordance with the present 
invention will now be described in detail with reference to accompanying F1G.3. 

[0055] F1G.3 illustrates a construction of the first embodiment of the content-based multimedia retrieval system in 
accordance with the present invention. 

20 [0056] As depicted in F1G.3, the content-based multimedia retrieval system comprises a color quantizer 300 for 
extracting a color histogram of a query image after being inputted the query Image by a user, a color space descriptor 
301 for describing color space infomiation which is basis of the extracted color histogram of the query image, a color 
quantization descriptor 302 for describing the color qunatization infomnation which is basis of the extracted color his- 
togram, a retrieval object system 305 for providing retrieval object image data 310-1 - 310-N and color histograms 

25 309-1 - 309-N of the retrieval object images, a color histogram converter 303 for converting the color histogram of the 
extracted query image so as to correspond to the color histograms 309-1 - 309-N of the retrieval object images (mul- 
timedia data to be retrieved) in order to perfomn a multimedia retrieval among the different systems, and a retrieval 
unit 304 for calculating similarity among the color histogram of the converted query image and the color histograms 

309- 1 - 309-N of the retrieval object images and outputting an image in accordance with the similarity as a retrieval 
30 result. 

[0057] The retrieval object system 305 connp rises image databases 31 0 for storing the retrieval object images (mul- 
timedia data to be retrieved) (data image 1 - image N) 310-1 - 310-N, color quantizers 308 for extracting the color 
histograms 309-1 - 309-N as the feature infonnation of the retrieval object image data (Image 1 ~ image N) 310-1 - 

31 0- N, feature databases 309 for storing the extracted color histograms 309-1 - 309-N of the retrieval object images, 
35 color space descriptors 306 for describing the color space inf omnation which is the basis of the color histogram of the 

retrieval object image, and color quantization descriptors 307 for describing the color quantization information which 
is the basis of the color histogram of the retrieval object image. 

[0058] Hereinafter, the operation of the content-based multimedia retrieval system will now be described in detail. 
[0059] First, when the query image is inputted, the color quantizer 300 extracts the color histogram which is the 
40 feature infomnation of the query image. 

[0060] Herein, the color histogram of the query image extracted by the quantizer 300 is extracted as a certain color 
space and a certain color quantization. 

[0061] The color space quantizer 301 describes the color space of the extracted color histogram, and the color 
quantization descriptor 302 describes the color quantization method of the extracted color histogram. 
^ [0062] Meanwhile, the retrieval object system 305 comprises the feature databases 309 for storing the color histo- 
grams having multimedia data structure, nannely, constructed with the same color spaces and same color quantization 
methods or different color spaces and different color quantization methods. 

[0063] Herein, the feature databases 309 are placed apart from each other, and are mainly connected to a networic 
such as the internet. 

so [0064] I n addition , the retrieval object systenns 305 to be the retrieval object (to own jointly image database or provide 
the image database to the retrieval unit) are systems for providing data on the internet circumstances (networi(), they 
can the same kind or different kind systems. 

[0065] Herein, the same kind of systems mean systems using color histograms constructed with same color spaces 
and color quantization methods, and the different kind systems mean systems using color histograrr^ constructed with 
55 different color spaces and color quantization methods. 

[0066] The color quantizers 308 inside of the retrieval object syst«n 305 extract the color histograms 309-1 - 309-N 
whfeh are the feature inf onmation of the plurality of image data (Image 1 - Image N) 31 0-1 - 31 0-N stored in the image 
database 31 0. In other words, the color quantizer 308 extracts the one color histogram 309-1 per the image data 31 0-1 . 
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[0067] The extracted color histogram 309-1 is connected to the pertinent image 310-1 , and is stored in the feature 
database 309. 

[0068] IHerein, the each feature database 309 stores the retrieval object images having the color histogranns con- 
structed with the different color spaces or different color quantization methods. 
5 [0069] The color space descriptors 306 describe the color spaces of the color histograms of the retrieval object 
images stored in the each feature database 309. and the color quantization descriptors 307 describe the qantizations 
of the color histograms of the stored retrieval object images. 

[0070] After that, the color histogram converter 303 finds out the color space and color quantization method (infor- 
mation) as the basis of the color histogram of the query image from the color space descriptor301 and color quantization 
10 descriptor 302. 

[0071] In addition, the color histogram converter 303 finds out the color space and color qunatization method as the 
basis of the color histograms of the retrieval object images from the color space descriptors 306 and color quantization 
descriptors 307 inside of the retrieval object system 305. 

[0072] After that, the color histogram converter 303 compares the color space and color quantization method (inf or- 
is mation) as the basis of the color histograms of the found retrieval object images with the color space and color quan- 
tization method (information) as the basis of the color histogram of the found query image, and converts the color 
histogram of the query image so as to con-espond to the color histograms of the retrieval object images when they are 
not same in the comparing result. 

[0073] In addition, the color histogram converter 303 can perform a function for converting the color histograms of 
20 the retrieval object images so as to correspond to the color histogram of the query image. 

[0074] For example, whenever the color spaces and color quantization methods as the basis of the color histogranns 
of the retrieval object images stored in the feature databases 309 are not same with the color space and color quan- 
tization method as the basis of the color histogram of the query image, the color histogram converter 303 converts the 
color histogram of the query image so as to correspond to the color histogranns of the retrieval object images. 
25 [0075] IHerein, when the color space and color quantization method of the color histogram of the query image is 
same with the color spaces and color quantization methods of the color histogranns of the retrieval object images (in 
the same kind system), the color histogram converter 303 provides the color histogram of the query image to the 
retrieval unit 307 as it is. 

[0076] On the contrary, when the color histogram of the query image and color histograms of the retrieval object 

30 images are constmc^ed with the different color spaces and color quantization methods each other (in the different kind 
system), the color histogram converter 303 converts the color histogram of the query image (color space and color 
quantization method whk;h are the basis of the color histogram of the query image) into the color space and color 
quantization method same with the color histogranris of the retrieval object images (color spaces or color quantizations 
method which are the basis of the color histograms of the retrieval object images), and provides the converted color 

35 histogram of the query image to the retrieval unit 304. 

[0077] As described above, the color histogram converter 303 converts one color histogram between the color his- 
togram of the query innage and color histogram of the retrieval object image so as to con-espond to the construction 
(color space and color quantization) of the other color histogram in order to compare the color histogram of the query 
image with the color histogranns of the retrieval object images. 

40 [0078] In other words, in retrieval of an image similar to the query image in the image database 310 storing the 
plurality of retrieval object images (multimedia data to be retrieved), the color histogram of the query image is compared 
with.the color histogranns 309-1 - 309-N of the image data stored in the feature database 309. 
[0079] Herein, the cok)r histogram converter 303 converts the cotor histogram of the query image and the color 
histogram of the retrieval object innage into the same histograms in order to compare the color histogram of the query 

45 Image with the color histogram of the retrieval object image. 

[0080] For example, when the user retrieves an image similar to the query image among the images to be retrieved 
constructed with the color histograms composed of different color spaces and different color quantization methods by 
different systems, whenever an Image similar to the query image is retrieved, the color histogram of the query image 
is converted into the color histogram of the same as the color histogram of the image to be retrieved. Accordingly, the 

so color histogram of the query image and the color histogram of the image to be retrieved can be compared. 

[0081] After that, the retrieval unit 304 outputs an image in accordance with the similarity as a retrieval result by 
comparing the color histogram of the query image or converted color histogram of the query image with the color 
histogranns of the retrieval object images (color histogranns of multimedia data to be retrieved) (caknjiating the similarity) 
on the color histogram converter 303. 

55 [0082] As described above, the content-based multimedia retrieval system in accordance with the present invention 
uses the color histogram converter 303, and can pertomn a multimedia retrieval between the different systems. 
[0083] FIG.4 illustrates a multimedia retrieval relations between different systems in accordance with the present 
invention. 
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[0084] As depicted in FIG.4, the local system A 400 comprises feature databases 401 for storing color histograms 
401-1 - 401 -N of images, a color space descriptor 402 for describing color space infonnation corresponding to the 
color histograms 401 -1 401 -N, and a color quantization descriptor 403 for describing color quantization infonnation. 
[0085] The local system B 404 comprises feature databases 405 for storing color histograms 405-1 - 405-N of \m- 
5 ages, a color space descriptor 406 for describing color space infonnation corresponding to the color histogranns 405-1 
- 405-N, and a color quantization descriptor 407 for describing color quantization infonnation. 
[0086] Hereinafter, the multimedia retrieval between the local systems A 400 and B 404 (different systems) will now 
be described in detail. 

[0087] First, the color space descriptor 402 inside of the local system A 400 describes the color space which is the 
10 basis of the color histogram as a RGB color space, and the color quantization descriptor 403 describes 5 levels of 
uniform quantization of the RGB color space. 

[0088] On the contrary, the color space descriptor 406 inside of the local system B 404 describes the color space 
which is the basis of the color histograms as a HSV color space, and the color quantization descriptor 407 describes 
7 levels of unifonn quantization of the HSV color space. 
15 [0089] Accordingly, when the comparison (retrieval) object image about the query image of the local system A is 
retrieved in the feature database 405 of the local system B 404, the local system A 400 can recognize the local system 
B 404. is different kind of system through the color space descriptors 402, 406 and color quantization descriptors 403, 
407. 

[0090] The local system A 400 can recognize the local system B is different kind of system, the color histogram 
20 (corresponding to the query image) based on the RGB color space (each 5 level uniform quantization) is converted 

into the color histogram (con-esponding to the retrieval object image) based on the HSV color space (each 7 level 

unifonn quantization), the converted color histogram is compared with the color histogram based on the HSV color 

space (each 7 level unifonn quantization), accordingly the multimedia data retrieval is possible. 

[0091 ] In other words, in order to compare the color histogram used in the local system A 400 with the color histogram 
25 used in the local system B 404, the one color histogram (color space and color quantization) between them is converted 

so as to con-espond to the construction of the other color histogram (color space and color quantization), accordingly 

the multimedia data retrieval is possible. 

[0092] In perfomrting the above-described cok)r histogram conversion, one quantum of the RGB color space can be 
projected to the HSV color space, and can be overtapped with at least one quantized subordinate space of the HSV 
30 color space. Herein, the degree of the projected color space can be found relatively and accurately by an integral 
calculus, however the caknjlatk>n is complicate, accordingly it is not advisable in embodinnent of a real-time system. 
[0093] In order to solve above-nnentioned problem, in the present invention, a histogram is converted quickly by 
using a sampling method, it will be described in detail later. 

[0094] Hereinafter, when the extract method of the color histogram of the query image and the color histogram of 
35 the retrieval object image is different in accordance with the color space and color quantization, the similarity is calcu- 
lated by comparing the two color histograms, it is judged whether the query image is similar to the retrieval object 
image in accordance with the calculated similarity, the above-described content-based multimedia retrieval nnethod 
will now be described in detail wrth reference to accompanying FIG.5. 

[0095] FIG.5 is a flow chart illustrating the first embodiment of a content-based multimedia retrieval method in ac- 
^ cordance with the present invention. 

[0096] First, when the query image is inputted to the multimedia retrieval system S500, it is retrieved whether the 
color histogram of the query image whk^h is same with the color histogram of the inputted query image is extracted 
before S501. 

[0097] When there is the color histogram extracted before, the color histogram value extracted before is read from 
45 the feature database S502. 

[0098] On the contrary, when the query is made newly by the user or the query image is made from the other image 
data and there is no color histogram extracted before, the color histogram about the query image inputted through the 
color quantizer 300 is extracted 8503. 

[0099] In the method for extracting the color histogram of the inputted query image, all pixels of the image are con- 
so verted into determined color space values, the converted color space values are divided into a plurality of subordinate 
color spaces in accordance with the color quantization, and it is recognized which subordinate color space transforms 
(displays) the each converted pixel value. 

[0100] After perfomning the process to the all pixels, it is possible to recognize how many pixels are projected to 
which subordinate color space. In other words, it is possible to know the values of the transofrmed bin number, and 
55 values found by an'anging the projected bin numbers with the first dimension vector is the color histogram. 

[0101] Herein, the each first vector element about the values arranged with the first vector is "Histogram Bin", and 
it means the subordinate color space. 

[0102] Accordingly, the bin number of the color histogram coincides with the color quantization number. 
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[01 03] After that, the color histogram is normalized regardless of the image size by dividing the first dimension vector 
value by a total image pixel number. 

[01 04] And , the color histograms values of the retrieval object images stored in the feature database are read S504. 
[01 05] Next, the color histogram of the query image extracted or read in the above process S502, S503 are compared 
5 with the color histograms of the all retrieval object Images stored in the feature database. 

[0108] In other words, the color space and color quantization method as the basis of the color histogram of the query 
image is compared with the color spaces and color quantization methods of the color histograms of the all retrieval 
object images S505. 

[0107] When the color space and color quantization method as the basis of the color histogram of the query image 
10 is compared with the color spaces and color quantization methods of the color histogranns of the all retrieval object 
images, and they are not same, the histogram converter 303 converts the color histogram of the query image so as to 
correspond to the color histogram of the retrieval object image. 

[0108] After that, the converted color histogram of the query image is compared with the color histogram of the 

retrieval object image in order to calculate the similarity S507. 
15 [0109] On the contrary, when the color space and color quantization method as the basis of the color histogram of 

the query image is compared with the color spaces and color quantization methods of the color histograms of the all 

retrieval object images and they are same, the similarity is calculated S507. 

[01 1 0] Herein, as the method for calculating the similarity, the known methods can be used. 

[01 11] In addition, the similarity is compared with a certain threshold value S508, and when the similarity is bigger 
20 than the certain threshold value, an image which corresponds to the color histogram of the image to be retrieved is 

regarded as the similar image and is outputted S509. 

[01 1 2] After that, a retrieval about the one query image is perfooned by perfomning the calculating process S507 for 
comparing and calculating the similarity of the color histograms of the all retrieval object images, when the color his- 
togram of the present retrieval object image which is compared and calculated the similarity presently Is not the color 

25 histogram of the last retrieval object image S51 0, the processes from the reading process S504 for reading the color 
histogram value of the retrieval object image stored in the feature database are perfonned repeatedly 
[0113] After that, when the calculating process for calculating the similarity is finished up to the color histogram of 
the last retrieval object image, it is judged whether any other query image is inputted S51 1 , when the other query image 
is inputted, the processes from the query image inputting process S500 are perionmed repeatedly- 

30 [01 14] Meanwhile, the color histogram converting method in accordance with the present invention transforms the 
each subordinate space (each quantum) of the color space of the quantized query image on the color space of the 
retrieval object image in order to convert the color histogram, and distributes the bin value on the proportion to the 
overiapped volume. The above-mentioned process will now be described in detail with reference to accompanying 
FIG.6. 

35 [01 15] FIG.6 illustrates a color histogram conversion method in accordance with the first embodiment of the present 
invention. 

[01 16] As depicted in FIG.6, the color space of the query image is described as the RGB color space 600, the RGB 
color space is quantized by dividing into 3 pieces about each component (RGB). In other words, 27 numbers of quan- 
tized subordinate spaces exist in the RGB color space, and the query image is descn'bed as the color histogram having 
40 27 bin size. 

[01 17] In addition, the color space of the retrieval object image is described as the HSV color space 601 , it is sepa- 
rately divided into 3 pieces about the S. and V components, the H is divided into 8 pieces or 6 pieces in accordance 
with region of a S value and is not divided in region adjacent to an achromatic color. In other words, there is 45 numbers 
of quantized subordinate spaces in the HSV color space, and the retrieval object images are descn'bed as a color 
^ histogram having 45 bin size. 

[0118] One quantum in the RGB color space is projected to the HSV color space, and can be overiapped with the 
at least one quantized subordinate space of the HSV color space. 

[01 19] In addition, a quantized subordinate space QSS_1 SODA of the RGB color space 600 is projected along quan- 
tized subordinate spaces TQSS_1 . TQSS_2, TQSS_9 of the HSV color space 601 601 A. 
50 [0120] Herein, a reference numeral 603 is a horizontal cross section of the HSV color space 601 , and a reference 
numeral 602 is a vertical cross section of the HSV color space 601 . 

[0121] Herein, when each volumetric ratio of the portion overiapped with the TQSS_1 , TQSS_2, TQSS_9 by the 
projected quantized subordinate space QSS_1 600A is 0.54, 0.35, 0.11 , a histogram value H_Q (QSS_1) of the quan- 
tized subordinate space QSS_1 (600A) is separately projected to histogram values H_T (TQSS_1), H_T (TQSS_2), 
55 H_T (TQSS_9) of the TQSS_1 , TQSS_2, TQSS_9 as much as 0.54 x H_Q (QSS_1 ). 0.35 x H_Q (QSS_1 ), 0. 1 1 x H_Q 
(QSS_1), 

[0122] In other words, the H_T (TQSS_1), H_T CTQSS_2), H_T (TQSS_9) are renewed as below Equation 1-3. 
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[Equation 1] 



H_T n"QSS_1) := H_T CrQSS_1) + H_Q (QSSJ) X 0.54 



5 



[Equation 2] 



H_T CTQSS_2) := H_T CTQSS_2) + H_Q (QSS„1) X 0.35 



[Equation 3] 



H_T (TQSS_9) := H_T n"QSS_9) + H_Q (QSS_1) X 0.1 1 



10 [0123] Herein, the H_T means a color histogram of the nomnalized retrieval object image, and the H_Q means a 
color histogram of the nomnalized query Image. 

[0124] In addition, the H_T (TQSS_k) describes a histogram value con^sponding to the index TQSS_k of the H_T. 
and the H_Q (QSSJ) describes a histogram value corresponding to the index QSSJ of the H_Q. 
[0125] When the above-mentioned process is pertomned about the all quantized subordinate space QSSJ of the 
15 RGB color space, the H_T (TQSS_k) values are renewed gradually, a histogram of the H_T (TQSS_k) found after the 
renewal is the color histogram which quantizes the HSV color space into 45 color grades (above-described quantization 
method) (Herein, I = 1 , 2, , 26. 27 and k = 1 , 2, , 44, 45). The above-described color histogram conversion process wilt 
now be described in detail with reference to accompanying F1G.7. 

[0126] FIG.7 is a flow chart illustrating a first embodiment of an operation process of the color histogram converter 
20 in accordance with the present invention. 

[0127] First, when the color spaces and quanttzatk>n methods of the query image and retrieval object image are 
different each other, the conversion of the color histogram starts S700. 

[0128] After that, the ail subordinate spaces QSSJ defined through the color space and quantization method as the 
basis of the color histogram of the query image are considered S701 . 
25 [0129] In other words, the all subordinate spaces QSSJ are projectedto the color space of the retrieval object Image, 
and a volumetric value VJ of the projected solid PV (QSSJ) is calculated S702. 

[0130] In addition, it is checked whether the all subordinate spaces QSSJ are overiapped with the subordinate 
spaces TQSS_k defined by the quantization method of the retrieval object image, when they are overiapped, a volu- 
metric value IV_kJ of a solid IV overiapped with the subordinate spaces TQSS_k and the projected solid PV (QSSJ) 
30 is calculated S703. 

[0131] After that, the all subordinate spaces TQSS_k including the overiapped volume among the quantized subor- 
dinate spaces of the color space of the retrieval object image are considered S704. 

[01 32] In the color histogram of the retrieval object image and color histogram of the query image, the all subordinate . 
spaces TQSS_k and all subordinate spaces QSSJ are separately renewed by below Equation 4 S705, accordingly 
35 the color histogram conversion is finished. 



40 [01 33] Herein, the IV_kJ is the volumetric value of the solid IV overiapped with the projected solid PV (QSSJ), the 
V J is a volumetric value of the solid PV (QSS_i) whk^h is projected to the color space of the retrieval object image. 
[0134] After that, when the renewal process is not performed up to a subordinate space TQSS__k among the all 
subordinate spaces TQSS_k S706, the processes from the considering process for considering the all subordinate 
spaces TQSS_k S704 are perfomned repeatedly. 

45 [0135] In addition, when the renewal process is not performed to the last subordinate space QSS_i among the all 
subordinate spaces QSSJ S707, the processes after the considering process S701 for considering the all subordinate 

spaces QSS-i are perfonmed repeatedly. ^ ^ ^ — — 

[01 38]^ Herein, a volumetric value of the projected solid can be found relatively and accurately by an integral cateulus, 
however the calculation is complk^ate, accordingly rt is diffrcult to adapt to a real-time system. 

50 [01 37] In order to solve the problem, in the present invention, the color histogram can be converted quickly by finding 
a volumetric value of the solid approximately by using a sampling method. 

[0138] The histogram conversion method using the sampling method will now be described in detail with reference 
to accompanying FIG.B. 

[0139] As depicted in FIG.B, 1 0 numbers of sampling points 800A are unifomnly extracted from the subordinate space 
55 QSSJ BOOB of the RGB color space BOO. 

[01 40] In other words, the transformation degree is caknjlated by finding out in which subordinate space of the HSV 
color space 802 the extracted 10 numbers of sampling modes exist. 

[01 41 ] Meanwhile, a reference numeral B01 A describes a total volume projected the subordinate space QSS_1 800B, 



[Equation 4] 



H_T (TQSS_k) : = H_T (TQSS.k) + H_Q (QSSJ) x I V_kJ / V J 
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and a reference numeral 801 B describes the sannpling points 800A are projected to the HSV color space. 

[01 42] In addition, a reference numeral 803 describes a horizontal cross section of the HSV color space 801 . and a 

reference numeral 802 describes a vertical cross section of the HSV color space 801 . 

[0143] Whenever the each sampling point 800A is projected to the HSV color space 801 . it is projected to the pertinent 
5 subordinate space as much as 0.1 x H_Q (QSS_1 ), when it is performed to the all 1 0 sampling points 800A, 5 sampling 
points are projected to the TQSS_1 as much as 0.5 X H_Q (QSS_1 ), it is projected to the TQSS_2 as much as 0.4 X 
H_Q (QSS_1), and it is projected to theTQSS_9 as much as 0.1 x H_Q (QSS_1). 

[0144] In other words, the H_T (TQSS_1 ), H_T (TQSS_2), H_T (TQSS_9) are renewed as below Equation 5-7. 
[Equation 5] H_T (TQSS_1 ) := H_T (TQSSJ ) + H_Q (QSS_1 ) X 0.5 

[Equation 6] H_T (rQSS_2) := H_T (TQSS_2) + H_Q (QSS_1) X 0.4 

15 

[Equation 7] H_T (TQSS.g) := H_T (rQSS_9) + H_Q (QSS_1) X 0.1 

[0145] Herein, the H_T is the color histogram of the normalized retrieval object image, and the H_Q is the color 
20 histogram of the normalized query image. 

[0146] In addition, the H_T (TQSS_k) is the histogram value corresponding to the index TQSS_k of the H_T, and 
the H_Q (QSS_i) is the histogram value corresponding to the index QSSJ of the H_Q. 

[0147] When the above-mentioned processes are perfomied to the all quantized subordinate space QSSJ of the 
RGB color space, the H_T (TQSS_k) values are renewed gradually, the histogram of the H_T (TQSS_k) found after 
25 the renewal is the color histogram which quantizes the HSV color space into 45 color grades (above-descn'bed quan- 
tization method) (Herein, i = 1 , 2, , 26, 27 and k = 1 , 2, , 44, 45). 

[0148] In the present inventton, in sampling of the quantized subordinate space of the color space of the query image, 
the number of the sampling point can be different in accordance with importance of the each subordinate space. 
[0149] In otherwords, by differentiating the number of the sampling point in accordance with importance of the each 
30 subordinate space, the present invention can improve the accuracy in conversion of the histogram, and at the same 
time can decrease the histogram conversion tinrte. 

[01 50] Herein, the importance of the subordinate space is the bin value of the histogram corresponding to the sub- 
ordinate space and the volumetric value of the subordinate space. 

[0151] First, in sampling of the subordinate space, the pertinent bin value is inputted from the color histogram, when 
35 the bin value is big, lots numbers of the sampling points are made, when the bin value is small, fewer numbers of the 
sampling points are made. 

[01 52] Because the subordinate space having 0 histogram bin value do not contribute to the histogram conversion, 
there is no need to perform the sampling, in particular. In the histogram of the minutely quantized color space there is 
pretty many beans having 0 bin value, accordingly rt is possible to decrease the histogram conversion time. 
40 [01 53] I n the bin value excluding 0, the reflection degree of the volumetric value of the quantized subordinate space 
of the color space of the query image is adjusted in proportion to the bin value. 

[01 54] In other words, as larger as the volume of the quantized subordinate space of the color space of the query 
image, lots of the sampling points are needed in order to cover the space, in sampling of the quantized subordinate 
space of the color space of the query inr>age, the nunnber of the sampling points is determined in consideration of the 

45 volumetric value of the subordinate space and bin value at the same time. 

[01 55] However, when the inriportance of the subordinate space is determined in order to determine the number of^ 
the sampling points, the present invention has no limit for a method considering the volun^tric value and bin value. 
[0156] The overall histogram conversion process including the determining process for determining the number of 
the sampling points of the quantized subordinate space of the color space of the query image in accordance with the 

50 importance of the volumetric value and bin value will now be described in detail with reference to accompanying F1G.9. 
[01 57] FIG.9 is a flow chart illustrating a second embodiment of an operation process of the color h istogram converter 
in accordance with the present invention. 

[0158] First, when the color spaces and quantization methods of the query image and retrieval object image are 
different each other, the conversion of the color histogram starts S900. 
55 [0159] In order to convert the color histogram, the all quantized subordinate spaces of the color histogram to be 
converted are considered S901 . 

[01 60] In other words, the nunnber of the sampling point (No_of_SP) about the quantized subordinate space QSSJ 
of the color space of the query image is determined in proportion to the histogram bin value and volumetric value of 
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the subordinate space QSSJ S902, and the sampling points are evenly extracted as same as the detemnined number 
from the subordinate space QSSJ S903. 

[0161] Herein, the each sampling point is described as a point value of the applied color space (RGB or HSV). 
[0162] In other words, when the RGB color space is applied, the sampling point is described as R=150, G=130, 
5 B=120. 

[01 63] In addition, after considering the extracted all sampling points (SP_i), rt is compared whether the color space 
as the basis of the color histogram of the query image (namely, the color space for extracting the sampling point) and 
the color space as the basis of the color histogram of the retrieval object image (the color space about the color his- 
togram to be converted) are same 3905. 

10 [01 64] When the color space as the basis of the color histogram of the query image and the color space as the basis 
of the color histogram of the retrieval object image do not con'espond, the color value of each sampling point SPJ of 
the all extracted sampling points is converted into the color space value of the retrieval object image S908. 
[0165] On the contrary, when the color space as the basis of the color histogram of the query image and the color 
space as the basis of the color histogram of the retrieval object image correspond, because only the color quantization 

IS method is different, the conversion process S906 for converting the color value into the color space value of the retrieval 
object image is abridged. 

[01 66] After that, the extracted all sampling points SPJ are projected to the color space of the retrieval object image, 
it is checked in which qunatized subordinate space TQSS of the color space of the retrieval object image the projected 
value is included S907, and the color histogram bin value con^esponding to the H_T (TQSS_k) is renewed as below 
20 Equation 8 S90B. 



[Equation 8] H_T (TQSS.k) := H T (TQSS.k) + H_G {QSS_i) X 1 / No_of_SP 

25 [01 67] Herein, the H_T means the color histogram of the nonnalized retrieval object image, and the H_Q means the 
color histogram of the normalized query image. 

[0168] In addition, the H_T (TQSS_k) nrteans the histogram value con^esponding to the index TQSS_k of the H_T, 
and the H_Q (QSSJ) means the histogram value corresponding to the index QSSJ of the H_Q. 
[0169] After that, when the sampling point SPJ which does not perfomi the renewal process S908 remains S909, 
30 the processes from the considering process S904 for considering the extracted all sampling points SPJ are perfomned 
repeatedly. 

[0170] After that, when the quantized subordinate space QSSJ which does not perform the renewal process S908 
remains S91 0, the processes from the considering process S901 for considering the all quantized subordinate spaces 
are performed repeatedly. 

35 [0171] Accordingly, the color histogram can be converted by performing the above-described processes about the 
extracted all sannpling points SPJ and all quantized subordinate spaces QSSJ. 

[0172] In an image retrieval, the color histogram conversion method using the above-described sampling nriethod 
can convert the color histogram into the different color space and different quantization, and it is adaptable when there 
is need to convert the histogram having the different quantization method Into the other quantization method about 
40 elements constructing not only the color but also a certain at least first dimension space. 

[01 73] Hereinafter, the other embodiment of the content-based multimedia retrieval system and n^ethod thereof will 
now be described in detail with reference to accompanying FIG.10 - 12. 

[0174] FIG.1 0 illustrates a second ennbodinDent of the content-based multimedia retrieval system in accordance with 
the present inventk>n. In other words, the cok>r histogram converter 303 is not comprised in the second embodiment 
^ of the content-based multimedia retrieval system, and a color histogram database 1 001 of the query image is com- 
prised. 

[0175] As described above, the content-based multimedia retrieval system comprises a color quantizer 1000 for 
extracting the color histogram of the query image by being inputted the query image from the user, a color histogram 
database 1 001 of the query innage for storing the extracted color histogram, a retrieval object system 1 003 {same as 

50 the 305 of FIG.3) for providing the retrieval object image data 1007-1 - 1007-N and the color histograms 1008*1 - 
1 008-N of the retrieval object images, and a retrieval unit 1 002 for calculating the similarity between the color histogram 
of the stored query image and color histogram of the retrieval object image and outputting an image according to the 
similarity as a retrieval result when the color histogram of the query image same with the inputted query image is 
quantized and extracted with the color space and quantization method con^esponding to the color histogram construc- 

55 tion of the retrieval object image, and is stored on the color histogram database 1 00 of the query image. The operation 
of the content-based multimedia retrieval system will now be described in detail. 

[01 76] First, when the query image is inputted, the color quantizer 1 000 extracts the color histogram as the feature 
information of the query image, and stores the extracted color histogram on the color histogram database 1 001 of the 
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query image. 

[0177] Herein, the color quantizer 1000 references the color space and quantization method as the basis of the 
retrieval object image from the retrieval object system 1003, and quantizes the color space and quantization method 
as the basis of the color histogram of the extracted query image so as to con^espond to the color space and quantization 

5 method of the retrieval object image. 

[0178] Accordingly, although the query image is same, the color histograms 1 001 -1 - 1 001 -N constructed with the 
different color spaces and quantization methods can be stored in the color histograms 1 001 -1 - 1 001 -N. 
[0179] Meanwhile, the present invention can extract the color histogram more quickly by adapting a method for 
finding the color histogram by reducing the size of the query image as a thumbnail image fonn. 

10 [0180] In addition, the quantization of the query image is perfonned in the retrieval object system 1 003, and it is also 
possible to compare and retrieval of the quantization result (color histogram information). 

[0181] Herein, the retrieval object system 1 003 to be the retrieval object (owning jointly the image database or pro- 
viding the image database to the retrieval unit) is a system providing data on the intemet circunnstances (network), it 
can be same kind or different kind system. 

IS [01 82] The color qu antizers 1 006 inside of the retrieval object system 1 003 extracts color histograms 1 008- 1-1 008-N 
as the feature information of a plurality of image data (Image 1 ....Image N) 1007-1 - 1007-N stored In the image 
database 1007. In other words, they extracts the one color histogram 1008-1 per the one image 1007. 
[0183] Herein, the extracted color histogranns 1008-1 - 1 008-N are connected to the pertinent images 1007-1 - 
1007-N, and are stored in the feature database 1008. 

20 [01 84] I n addition , the retrieval object system 1 003 comprises a color q uantizer 1 006 for extracting the color histogram 
of the retrieval object image, color space descriptors 1004 for describing the color space information as the basis of 
the color histogram stored in the feature database 1 008 after being extracted by the color quantizer 1 006, and color 
quantization descriptors 1005 for describing the color quantization information as the basis of the stored color histo- 
gram. 

25 [01 85] In addition , the retrieval object system 1 003 is constructed with the plurality of feature databases 1 008, images 
having color histograms constructed with different color spaces and different color quantizations are stored in the each 
feature database 1 008, the color space and quantization of the color histogram can be described by color space quan- 
tizers 1004 and color quantization descriptors 1005. 

[0186] Herein, the each feature database 1008 is placed apart from each other, and is connected with a nctworic 
30 such as the intemet. 

[0187] After that, the retrieval unit 1 002 judges whether the color histogram of the query image same with the query 
image inputted by the user is quantized by the color space and quantization method corresponding to the color histo- 
gram construction of the retrieval object image, is extracted, and is stored in the color histogram database 1 001 of the 
query image. 

35 [0188] In the judging process, when the color histogram of the query image same with the query image inputted by 
the user is stored in the color histogram database 1 001 of the query image, the retrieval unit 1 002 caknjiates similarity 
between the color histogram of the stored query image and color histogram of the retrieval object image, and outputs 
an image in accordance with the similarity as a retrieval result. 

[01 89] On the contrary, when the color histogram of the query image same with the query image inputted by the user 
^ is not stored in the color histogram database 1 001 of the query image, the retrieval unit 1 002 extract the colo r histogram 
of the inputted query image, calculates similarity between the color histogram of the stored query image and color 
histogram of the retrieval object image, and outputs an image in accordance with the similarity as a retrieval result. 
[0190] Herein, the color histogram of the extracted query image is stored in the color histogram database 1 001 of 
the query image. 

^ [01 91 ] Hereinafter, the second embodiment of the content-based mu Itlmedia retrieval method perfomied by the con- 
tent-based multimedia retrieval system of FIG.1 0 will now be described in detail with reference to acconnpanying FIG.1 1 . 
[0192] FIG.1 1 is a flow chart illustrating the second embodiment of the content-based multimedia retrieval nrtethod 
in accordance with the present invention. 

[0193] Rrst, when the query image is inputted S1 100, the quantizer 1000 (or retrieval unit 1002) acquires the color 
so space description infomnation and quantization descriptbn information of the retrieval object Image S11 01 . 

[01 94] After that, the quantizer 1 000 (or retrieval unit 1 002) extracts the color histogram of the query image by quan- 
tizing the color histogram of the query image as the color space and quantization method corresponding to the color 
histogram construction of the retrieval object image by referencing the color space description infomoation and quan- 
tization description information S1102. 
S5 [0195] After that, the color histogram values 1008-1 - 1 008-N of the retrieval object images stored in the feature 
database 1008 are read S1 103, the similarity between the extracted color histogram of the query image and color 
histogram of the read retrieval object image are compared S1 104. 

[01 96] When the similarity of the color histogram of the query image and color histogram of the retrieval object image 
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is larger than a certain threshold value S1105, the retrieval object image Is outputted as a similar Image S1106. 
[0197] On the contrary, when the similarity of the color histogram of the query image and color histogram of the 
retrieval object image is smaller than a certain threshold value, the outputting process S1 1 06 for outputting the retrieval 
object image (multimedia data to be retrieved) as the similar image is abridged. 
5 [01 98] After that, in the similarity comparing process, it is judged whether the comparing process is perfomied up to 
the last retrieval object Image (Image N) S1107. 

[0199] When the comparing process for comparing the similarity to the last retrieval object Image (Image N) is not 
finished, it is judged whether the histogram construction about the next retrieval object image inside of the retrieval 
object system 1003 is differentiated from former S1109. 
10 [0200] When the histogram construction about the next retrieval object image is differentiated from the former, and 
the processes from the acquiring process S11 01 for acquiring the color space description infomiation and quantization 
description infomnation of the retrieval object image are perfomied repeatedly. 

[0201 ] On the contrary, when the histogram construction about the next retrieval object image is same with the former, 
the processes from the reading process S1103 for reading the color histogram values 1008-1 - 1008-N about the 
15 retrieval object image are performed repeatedly. 

[0202] Meanwhile, when the process for comparing the similarity is performed up to the last retrieval object image 
(Image N), It is judged whether the other query image is inputted S1108. 

[0203] When the other query image is inputted, the processes from the inputting process S1100 for inputting the 
query image are performed repeatedly, when the other query image is not inputted, the multimedia data retrieval is 
20 finished. 

[0204] Hereinafter, a third embodiment of the content-based multimedia data retrieval method perfonrted by the 
system of FIG. 10 will now be described in detail with reference to accompanying FIG.12. 

[0205] FIG.12 is a flow chart illustrating the third embodiment of the content-based multimedia retrieval method In 
accordance with the present invention. The processs for storing the color histogram of the query image and the process 
25 for using the stored color histogram to the multimedia retrieval are same with the second embodiment of FIG.11. 
Accordingly, hereinafter, the processes S1 100. S1 101. S1 103 - S1 109 same with FIG.11 are described with same 
reference numerals, and overiapped descriptions are abridged. 

[0206] First, when the query image is inputted S1 1 00, by referencing the color space and color quantization descrip- 
tion infonmation of the retrieval object image S1101, it is judged whether the color histogram same with the inputted 
30 query innage is quantized so as to correspond to the color histogram construction of the retrieval object image, is 
extracted, and is stored in the color histogram database 1001 of the query image SI 200. 

[0207] When the color histogram of the query image is extracted before and is stored in the color histogram database 
1001 of the query image, the color histogram value of the stored query image Is read SI 202, and the processes from 
the reading process S1103 for reading the color histograms values 1008-1 - 1008-N about the retrieval object image 
35 are performed. 

[0208] On the contrary, when the color histogram of the query image Is not stored in the color histogram database 
1001 of the query Image after extracting before, the color histogram of the inputted query image is quantized with the 
cx>lor space and quantization nnethod corresponding to the color histogram construction of the retrieval object image, 
and is extracted SI 201 . 

^ [0209] After that, the cwlor histogram of the extracted query image is stored in the color histogram database 1 001 
of the query image SI 203, the prcx^esses from the reading proc:ess S1 103 for reading the c:olor histogranns values 
1 008-1 - 1008-N about the retrieval object image are performed. 

[021 0] As described above, since the color histogram of the query image corresponding to the color histogram con- 
struction of the image to be retrieved is stored in advartce. the present invention can skip a new proc^ess for extracting 
^ the color histogram of Input query Image when a query image same as the stored color histogram of the query image 
is inputted next time. Acxordingly. the present invention c^an improve the multimedia retrieval speed. 
[0211] As descn'bed above, the present invention is capable of performing an interoperable multimedia retrieval 
between different systems using the color histogram constructed with different color spaces and color quantization 
methods. 

so [0212] In addition, the present invention is capable of retrieving an image to be retrieved corresponding to the query 
image of itself in a database of the other party, when retrieving multimedia data in the systen:is using different color 
histogram constructed with different color spaces and color quantization methods. 

[0213] In addition, the present invention is enable of retrieving images by connecting databases storing various and 
different and wide range of images on a networic environment such as Internet. 
55 [0214] In addition, the present invention is c^apable of improving the multimedia retrieval speed by extracting and 
storing the color histogram of the query Image in acivance. 
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Claims 

I , A content-based multimedia retrieval system, comprising : 

5 a first color quantization means for extracting a color histogram of query multimedia data ; 

a second color quantization means for extracting color histograms of multimedia data to be retrieved ; and 
a histogram conversion means for converting the color histogram of the extracted query multimedia data and 
the color histogram of the multimedia data to be retrieved so as to be same each other. 

10 2. The content-based multimedia retrieval system according to daim 1 . wherein the multimedia data are image data 
or video data. 

3. The content-based multimedia retrieval system according to claim 1 , wherein the color histograms are constructed 
on the basis of color spaces and color quantization methods. 

15 

4. The content-based multimedia retrieval system according to daim 1 , wherein the content-based multimedia re- 
trieval system further comprises a description means for describing color space and color quantization infomriation, 
which are the bases of the color histograms, wherein the description means comprises : 

20 a color space description means for describing color space constmcting the color histogram ; and 

a quantization description means for describing color quantization method constructing the color histogram. 

5. The content-based multimedia retrieval system according to claim 1 , wherein the histogram conversion means 
converts the color histogram of the query multimedia data so as to be corresponding to color space and color 

25 quantization method of the multimedia data to be retrieved. 

6. A content-based multimedia retrieval method, comprising : 

inputting query multinDedia data ; 
30 converting the color histogram of the inputted query multimedia data and the color histogram of the multimedia 

data to be retrieved so as to be same each other; and 

calculating a similarity between the query multimedia data and multimedia data to be retrieved on the basis 
of the converted color histogram and outputting a retrieval result in accordance with the calculated similarity. 

35 7. The content-based multimedia retrieval method according to daim 6, wherein the multimedia data is image data 
or video data. 

8. The content-based multimedia retrieval method according to daim 6, wherein the color histograms are constructed 
on the basis of color spaces and color quantization methods. 

40 

9. The content-based multimedia retrieval method according to claim 6, wherein the converting process for converting 
into the same histogram comprises 

judging whether the color histogram of the query multimedia data is extraded before ; 
45 reading a color histogram value extraded before and identifying based color space and color quantization 

method ; 

reading a color histogram value of the multimedia data to be retrieved and identifying based color space and 
color quantization method ; and 

converting the color histogranDS into the color histogranns of the same color space and color quantization 
so method when the color histogram of the query multimedia data and the color histogram of the multimedia data 

to be retrieved are not same in comparing. 

10. The content-based multimedia retrieval method according to claim 9, wherein the content-based multimedia re- 
trieval method further comprises extracting a color histogram of the inputted query multimedia data when the color 

55 histogram of the query multimedia data is not extraded before. 

II . The content-based multimedia retrieval method according to claim 6, wherein the process for converting into the 
same histogram is perfomned by ref erendng the color space description information and quantization description 
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information of the muttimedia data to be retrieved and query multimedia data. 

1 2. The content-based muitimedia retrieval method accordin g to claim 6, wherein the process for outputti ng the retrieval 
result comprises: 

5 

comparing the calculated similarity with a certain threshold value; and 

outputting multimedia data conresponding to the color histogram of the muitimedia data to be retrieved as a 
similar muitimedia data when the similarity is larger than the certain threshold value. 

10 13. A content-based multimedia retrieval method for retrieving muitimedia data by comparing query multimedia data 
with multimedia data to be retrieved, comprising : 

extracting a color histogram of the query multimedia data ; 
extracting a color histogram of the multimedia data to be retrieved ; 
15 comparing the extracted color space and color quantization method of the query image with the color space 

and color quantization method of the multimedia data to be retrieved ; and 

converting the color histogram of the extracted query muttimedia data or the multimedia data to be retrieved 
into the same color histogram so as to have same color space and color quantization method each other when 
the color spaces and color quantization methods of the extracted muttimedia data and multimedia data to be 
20 retrieved are different each other, and perfonning a retrieval in accordance with the similarity between the 

query multimedia data and multimedia data to be retrieved. 

14. The content-based multimedia retrieval method according to claim 13, wherein the content-based muttimedia re- 
trieval method further comprises a step of perfonning a retrieval in accordance with the similarity between the 

25 extracted query multimedia data and muitimedia data to be retrieved when the color space and color quantization 

method of the extracted query multimedia data are same as the color space and color quantization method of the 
muttimedia data to be retrieved. 

1 5. The content-based multimedia retrieval method according to claim 13, wherein the converting process for convert- 
30 ing the color histogram converts the color space and color quantization method of the query muttimedia data so 

as to correspond to the color space and color quantization method of the multimedia data to be retrieved. 

1 6. The content-based muttimedia retrieval method according to daim 1 3, wherein the process for converting the color 
histogram converts the color space and color quantization method of the multimedia data to be retrieved so as to 

35 correspond to the color space and color quantization method of the query multimedia data. 

17. A content-based multimedia retrieval method, comprising : 

comparing the color spaces and color quantization methods of the query multimedia data and multimedia data 

40 to be retrieved ; 

converting the color histogram of the query multimedia data or color histogram of the multimedia data to be 
retrieved when the color space and color quantization method of the query multimedia data and the color 
space and color quantization method of the multimedia to be retrieved are different; and 
calculating a similarity between the converted or unconverted query muttimedia data and multimedia data to 

45 be retrieved, and perfonning a retrieval in accordance with the calculated similarity. 

18. The content-based multimedia retrieval method according to claim 17, wherein the color histogr£im converting 
process converts the color histogram of query multimedia data so as to correspond to the color space and color 
quantization method of the muttimedia data to be retrieved. 

50 

19. A multimedia data generation method in retrieval of the muttimedia data using a color histogram of the muttimedia 
data as a feature element for a content-based muttimedia data retrieval, comprising : 

generating color space description infonnation describing what color space the color histogram of the multi- 
55 media data is constructed; and 

generating quantization description information describing what color quantization method the color histogram 
of the muttimedia data is constructed. 
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20, The multimedia data infomnation generation method according to claim 19, wherein the quantization description 
information describes a quantization level for each component of the color space, and a quantization method for 
each component, wherein the quantization method is described uniformly or non-unrfomnly. 

5 21 . A histogram conversion method of multimedia data in retrieval of the multimedia data using a color histogram of 
the multimedia data as a feature element for a content-based multimedia data retrieval, comprising : 

projecting each subordinate color space of the color space of the quantized query multimedia data on the color 
space of the multimedia data to be retrieved ; and 
10 distributing a bin value to each quantum of the color space of the multimedia data to be retrieved, which is 

overiapped with the projected color spaces. 

22. The histogram conversion method according to claim 21 , wherein the bin value is distributed in proportion to volume 
projected and overiapped to each quantum of the color space of the multimedia data to be retrieved. 

15 

23. The histogram conversion method according to claim 21 , wherein the bin value distributing process comprises: 

extracting a certain nunnber of sampling points in the color space of the query multimedia data; and 
approximating the extracted sampling point with a ratio distributed to each subordinate space of the projected 
20 multimedia data to be retrieved, 

wherein the number of the sampling point is detemnined in accordance with the importance of the quantized 
subordinate space of the color space of the query multimedia data. 

24. The histogram conversion method according to claim 23, wherein the importance is detennined by considering a 
25 volumetric value of the quantized subordinate space of the color space of the query multimedia data or a bin value 

of the color histogram of the query multimedia data. 

25. The histogram conversion method according to daim 23, wherein the importance is detennined by considering 
both a volumetric value of the quantized subordinate space of the color space of the query multimedia data and 

30 a bin value of the color histogram of the query multimedia data. 

26. The histogram conversion method accordingto daim 25, wherein the importance is determined by using a threshold 
value which is set as a value not sampling the bin value *0' or a value sampling the bin value excluding '0* with a 
maximum number. 

35 

27. The histogram conversion method according to claim 26, wherein the histogram conversion method adjusts de- 
flection degree of a volumetric value of the quantized subordinate space of the color histogram of the query mul- 
timedia data about the bin value exduding '0*. 

^ 28. A content-based multimedia retrieval system, comprising : 

a first color quantization means for extracting color histogram of the multimedia data to be retrieved ; 

a description means for descn'bing color space and color quantization method of the extracted color histogram ; 

and 

^ a second color quantization means for extracting color histogram of a query multimedia data with a method 

same as the described color space and color quantization method in order to perform the multimedia data 
retrieval. 

29. The content-based multimedia retrieval system according to claim 28, wherein the content-based multimedia re- 
50 trieval system further comprises a retrieval unit for calculating a similarity between the color histogram of the query 

multimedia data extracted before and the color histogram of the multimedia data to be retrieved, and outputting 
multimedia data in accordance with the calculated similarity as a retrieval result. 

30. The content-based multimedia retrieval system according to claim 29, wherein the content-based multimedia re- 
55 trieval system further comprises a database for storing the color histogram of the extracted query multimedia data. 

31 . The content-based multimedia retrieval system according to claim 29, wherein the description means comprises : 
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a color space description means for describing color space information which is the basis of the color histogram 
of the extracted multimedia data to be retrieved ; and 

a quantization description means for describing color quantization information which is the basis of the color 
histogram of the extracted multimedia data to be retrieved. 

5 

32. A content-based multimedia retrieval method, comprising : 

judging whether the color histogram of the query multimedia data corresponding to the color space and quan- 
tization method of the multimedia data to be retrieved is stored before ; and 
10 calculating a similarity between the color histogram of the stored query multimedia data and the color histogram 

of the multimedia data to be retrieved and perfomning a muttimedia retrieval in accordance with the calculated 
similarity. 

33. The content-based multimedia retrieval method according to claim 32, wherein the content-based multi- 
media retrieval method further comprises : 

15 

quantizing and extracting the query multimedia data with the color space and color quantization method 
of the multimedia data to be retrieved when the color histogram of the query multimedia data is not stored 
before; 

storing the color histogram of the quantized and extracted query multimedia data; and 
20 calculating a similarity between the color histogram of the extracted query multimedia data and the color 

histogram of the multimedia data to be retrieved and performing a muttimedia retrieval in accordance with 
the calculated similarity. 
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